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Cooper’s Taxonomy of Literature Reviews 
Applied to Meta-Analyses in Educational Achievement 



The quantity of published research in most disciplines is quite overwhelming. In education, an 
average of 1,500 journal articles and 1,100 documents are added each month to the ERIC 
Database, totaling 18,000 journal articles and 13,000 documents annually (ERIC, 1993). It is 
impossible for anyone to read not to mention synthesize this vast amount of literature. Moreover, 
keeping abreast in a particular topic area or areas can be unmanageable since conflicting outcomes 
of individual studies may complicate reading and synthesizing reports. Over time, researchers have 
developed various methods to combine evidence from different studies, some of which include: 
the narrative review, vote counting methods, combined significance tests, and quantitative 
synthesis (Light & Pillemer, 1984). Meta-analysis is one of the most recent advances in 
quantitative synthesis of research. 

Glass first coined the term meta-analvsis in 1976 (Glass, 1976) and hundreds now exist in 
the educational, psychological, and medical literatures. This paper is one component of a larger 
project which identified and analyzed meta-analyses in education related to achievement. In this 
paper. Cooper’s taxonomy of literature reviews (Cooper, 1988) was applied to the meta-analyses. 

Perspective 

Cooper has developed a taxonomy that classifies literature reviews based on six 
characteristics. These characteristics include: “focus of attention, goal of the synthesis, 
perspective on the literature, coverage of the literature, organization of the presentation, and 
intended audience” (Cooper and Hedges, 1994, p. 4). These six characteristics allow research 
syntheses to be distinguished from one another. A brief description of each characteristic as 
defined by Cooper (1988) will follow. 
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Focus 

The material that is of primary concern to the author determines the focus of a review. 
Most reviews in education usually concentrate on one or more of the following areas: a) research 
outcomes, b) research methods, c) theories, and/or d) practices or applications. 

Goals 

The end objective for the reviewer establishes the goal of the review. The most common 
goal in literature review is the integration of previous literature. This can include: a) 
generalization, b) conflict resolution in which a new “conception” of an idea is offered as an 
explanation of contradictory ideas or statements, and c) linguistic bridge building that bridges the 
gap between theories to create a “common linguistic framework” (Cooper, 1988, p. 108). A 
second goal is criticism in which the reviewer demonstrates that conclusions about the literature 
are biased or incorrect A third goal is identification of central issues. 

Perspective 

Perspective pertains to the tone of the discussion section. Reviewers either attempt to 
provide a neutral perspective that involves exposing many sides to an issue or espouse a position 
that may involve limiting the information presented. 

Coverage 

The process of identifying literature included in the review determines the nature of the 
coverage. The coverage can be a) exhaustive, b) exhaustive with selective criterion, c) 
representative of core material, and/or d) central or pivotal to the reviewer’s goal. 

Organization 

Organization concerns the arrangement of the findings and conclusions of the review. 
These categories include: a) historically, b) conceptually, and/or c) methodologically. 
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Audience 

The intended audience of the review can include: a) specialized scholars, b) general 
scholars, c) practitioners or policy makers, and/or d) the general public. Cooper suggests that the 
intended audience is identified through the author’s style of writing. 

Methods 

The methods used in this project were similar to the stages of research synthesis suggested 
by Cooper and Hedges (1994) which encompassed problem formulation, data collection, data 
evaluation, analysis and interpretation. Data collection included the literature search process 
described below. Data evaluation consisted of coding the meta-analyses using a coding form and 
manual developed for this project. One hundred and three meta-analyses were coded according to 
Cooper’s taxonomy defined below. 

Operational Definitions for Coding 
I. Focus 

The purpose section of the meta-analysis was the source for the type of focus to determine 
the central interest of the reviewer. Occasionally the problem statement or discussion section also 
provided an indication of the author’s central interest. Meta-analyses may contain multiple foci. 
The first two foci are self-explanatory. 

Research outcomes. 

2) Research methods. 

3) Theories. Specific theory identified within the purpose or included in the discussion 

section. 

4) Practices or applications. Specific applications of the research identified in the purpose. 
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II. Goals 

The goals were identified from the purpose statement as well as the literature review 
section. The goals were the author’s aspirations for the review. Meta-analyses may contain more 
than one goal. 

11 Integration. A review was considered to be integrative if the author compared the 
results from several studies in the review of literature or background sections. 

a) Generalization. General statements from multiple specific instances. 

b) Conflict resolution . Identification of conflict among multiple primary studies. 

c) Lin guistic bridge building . Linking theories and concepts to practice. 

2) Criticism. The literature review or purpose statement contained critical statements 

concerning past conclusions. 

3) Identification of central issues. The literature review and/or problem statement 
identified specific issues. The three types listed below are self-explanatory. 

a) Questions that have dominated past endeavors. 

b) Questions that should dominate future endeavors. 

c) Methodological problems that have prevented a topic from progression. 

III. Perspective 

Information concerning the perspective was found in the discussion section. 

1) Neutral representatio n. Alternative explanations for the results were presented. 

2) Espousal of a position. No alternative explanations for the results were offered. 

IV. Coverage 

Type of coverage was determined from the data sources described in the methods section. 
Meta-analyses may contain multiple types of coverage. 
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1) Exhaustive. Included entire literature or most of it with virtually no exclusion criteria. 

Contained a literature searcn of at least 2 databases. 

2) Exhaustive with selective criterion. Contained a literature search of at least 2 databases. 

Contained exclusion criteria which eliminated portions of the literature. 

3) Representative. Only used one journal or one source for the primary studies. 

4) Central or pivotal. Contained primary studies from previous reviews or meta-analyses. 

V. Organization 

The order in which the results were presented determined the organization. Meta-analyses 
could be organized in more than one way. 

1) Historically. Chronological order. 

2) Conceptually. Results with similar concepts were grouped together. 

3) Methodologically. Results were grouped according to methodological features. 

VI. Audience 

The audience was determined by the type of journal the meta-analysis was published in, 
the purpose, the implications, as well as the writing style of the author. Meta-analyses may have 
more than one audience. 

1) Specialized scholars. Journal was of limited circulation. Readership was from a special 

interest group. 

2) General scholars. Journal was of wide circulation. Readership was broad and from 

many disciplines. 

3) Practitioners or policy makers. Readership included practitioners and policy makers. 

Implications were practice-based. 

4) General public . Results were stated in terms general public could understand. 




V 
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Data Source 

Using the suggestions by Cook et al. (1992) and Cooper (1989), the literature search 
procedures were conducted in several steps that included: computerized database searching of 
ERIC, PsycLIT, and Medline; ancestry; invisible college; and hand searching of Review ^ 
FHnP..tinn«l Research . The keywords: (meta-analytic" or "meta analytic" or "meta-analysis or 
"meta analysis" or "quantitative synthesis" or “Best Evidence Synthesis") and ("education" or 
"coaching" or "training" or "teaching" or "achievement") and "Language = English," identified 
1197 citations. Once citations were identified, titles and abstracts were read to determine if 
retrieval was necessary. A total of 694 documents were retrieved. 

Criteria for Inclusion of Meta-A nalytic Studies 

The criteria for inclusion of meta-analyses in the study were: 

1. Published journal articles for the years 1984-1993; [Several authors recommend 
inclusion of unpublished studies in meta-analysis since unpublished studies are more likely to have 
nonsignificant results, and consequently, lower effect sizes (Cook et al., 1992; Glass, McGaw, & 
Smith, 1981; Rosenthal, 1991). However, Cooper, Dorr, & Bettencourt (1995) found published 
and unpublished meta-analyses differed by no more than 0.04 standard deviations in effect size.] 

2. published research reports of meta-analyses when the corresponding meta-analysis was 

not included in the database; 

3. meta-analyses with at least one outcome measure of achievement; and 

4. at least one reported effect size or statistic which could be converted into an effect size, 

Criteria for Exclusion of Meta-A nalvtic Studies 

The criteria for exclusion of studies from the meta-synthesis were. 
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1. Outcomes with higher education programs in certain fields of study (e.g., medical, 
nursing, dental) since these represent areas in a specialized knowledge base; 

2. outcomes with preschoolers because achievement measures are different at this level; 

3. aptitude outcomes since these measure the ability to perform rather than achievement. 




and 

4. interventions with high risk infants. 

Four hundred twenty-seven meta-analyses were among the citations retrieved. One 
hundred seventeen meta-analyses were published prior to 1984 and thus did not meet criteria for 
inclusion in the study. One hundred ninety-one published meta-analyses did not have outcome 
measures related to achievement and were also not included in the study. The remaining 119 
published meta-analyses were related to achievement and identified for possible inclusion in the 
study. Of these, 16 meta-analyses were excluded based on the exclusion criteria listed above (see 
Appendix B). Therefore, 103 published meta-analyses were included in the study (see Appendix 
A). 

Results and Conclusions 

The results are presented in Tables 1-6. Research outcomes were the focus in a majority of 
the mi ta-analyses while research methods and theory building or assessment were the focus in less 
than 10% of the meta-analyses (see Table 1). Generalization was a goal of all the meta-analysts 
while linguistic bridge building which deals with theories was a goal in 12% of the meta-analyses 
(see Table 2). The perspective of the meta-analysts was fairly evenly divided between neutral and 
espousal of a position (see Table 3). Exhaustive coverage with selective criteria was the norm for 
the majority of the meta-analyses (see Table 4). The findings in all of the meta-analyses were 
arranged conceptually. The findings were also arranged methodologically in slightly less than half 
of the meta-analyses (see Table 5). The intended audience was fairly evenly divided between 

d 




specialized and general scholars. In addition, pr;:Ctitioners were the intended audience in over 
90% of the meta-analyses (see Table 6). 

Implications 

As suggested by Cooper (1988), using the taxonomy of reviews can benefit several 
audiences. First, and probably most important, the taxonomy can help readers of meta-analyses 
assess study quality. Secondly, the taxonomy can provide a framework for meta-analysts who are 
conducting and publishing meta-analyses. Third, a taxonomy can assist journal editors in assessing 
the merits of meta-analyses. 

Table 1 



Focus of the Meta-Analvses 



Focus 


n 


Percent 


Research outcomes 


101 


98.1 


Research methods 


10 


9.7 


Theories 


10 


9.7 


Practices 


78 


75.7 



Note. Meta-analyses may contain multiple foci. 



Table 2 



Goals of the Meta- Analysis 



Goals 


n 




Percent 




Generalization 


103 




100 




Conflict resolution 


43 




41.7 




Bridge building 


13 




12.6 




Criticism 


16 




15.5 




IdentiHcation issues 


1 




1.0 




Note. Meta-analvsis mav contain multiple goals. 








Table 3 










Perspective of the Meta- Analyses 










Perspective 


n 


Percent 






Neutral 


54 


52.4 






i .spousal 


49 


47.6 






Table 4 










Coverage of the Meta-Analvses 










Coverage 


n 




Percent 




Exhaustive - comprehensive 


0 




0.0 




Exhaustive - selective 


87 




84.5 




Representative 


4 




3.9 




Central 


10 




9.7 





Note. Meta-analyses may contain more multiple types of coverage. 
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Table 5 

Organization of the Meta-Analyses 
Organization 



Historically ^ 

Conceptually 

Methodologically 

N,,te. Meta-analyses may be organized in more than one way. 



n 


Percent 


61 


59.2 


59 


57.3 


94 


91.3 


4 


3.9 



Table 6 

Audience of the Meta-Analvses2 
Audience 

Specialized scholars 
General scholars 
Practitioners 
Public 

Note. Each meta-analysis can be coded into more than one category. 
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